Skip to content

docs(harnesses): summarize the 2026 landscape of AI agent harnesses (closes #95) - #174

Merged
d-morrison merged 8 commits into
mainfrom
docs/ai-harness-landscape
Sep 1, 2026
Merged

docs(harnesses): summarize the 2026 landscape of AI agent harnesses (closes #95)#174
d-morrison merged 8 commits into
mainfrom
docs/ai-harness-landscape

Conversation

@d-morrison

@d-morrison d-morrison commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Important

Merge #173 first --- this PR is stacked on its branch (docs/claude-code-other-models) because the new section cross-references @sec-ai-claude-code-other-models. GitHub retargets this PR to main when #173 merges.

Closes #95

Adds a coding-agents section, chapters/ai-tools/ai-harness-landscape.qmd, mapping the harness landscape as of August 2026 and condensing the research summary in #95:

  • the model-plus-harness framing and benchmark saturation
  • terminal-first and editor-integrated harnesses (two linked tables), the autonomous and cloud tier, and the minimalist counter-trend
  • the open-source versus proprietary split, local-model stacks, and the general agent-framework layer
  • a job-to-harness selection table

Every volatile figure, ownership claim, and corporate event is dated and attributed to the summary in a leading callout. The adjacent inconsistency about Gemini CLI in two older fragments is tracked as #175 rather than fixed here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Collaborator Author

Round 1 disposition, pushed as b645180: Addressed. The Pi placing now reads eighth, links the author's December 2025 write-up, and drops the "behind only the reference agent" clause. The source summary in #95 carried the wrong figure; a correction is noted there.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Base automatically changed from docs/claude-code-other-models to main September 1, 2026 20:06
@d-morrison
d-morrison force-pushed the docs/ai-harness-landscape branch from b645180 to 3601dc6 Compare September 1, 2026 20:07
Copilot AI lite review requested due to automatic review settings September 1, 2026 20:07
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

It introduces broken relative links and a lychee exclusion that doesn’t match the linked URL form, plus a factual inconsistency with the cited issue #95.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds a new “Harness Landscape in 2026” documentation section to the wai Quarto site, summarizing the August 2026 state of AI agent harnesses and how to choose among them (per issue #95).

Changes:

  • Add a new harness-landscape chapter fragment and include it from chapters/coding-agents.qmd.
  • Extend the spelling WORDLIST with new proper nouns/terms used in the added section.
  • Add a lychee link-check exclusion for Amp’s homepage.
File summaries
File Description
lychee.toml Adds an exclusion intended to avoid lychee failures on Amp’s homepage.
inst/WORDLIST Adds new allowed words used by the new harness-landscape section.
chapters/coding-agents.qmd Adds a new top-level section and includes the new landscape fragment.
chapters/ai-tools/ai-harness-landscape.qmd New content describing the 2026 harness landscape, including comparison tables and selection guidance.
Review details

Suppressed comments (1)

chapters/ai-tools/ai-harness-landscape.qmd:160

  • Same relative-path issue as earlier in this file: agent-orchestration.qmd is one directory up from chapters/ai-tools/. This link will be broken as written.
[when orchestration helps](agent-orchestration.qmd#sec-orch-when) makes the same case for this lab's own use.
  • Files reviewed: 4/4 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread chapters/ai-tools/ai-harness-landscape.qmd
Comment thread lychee.toml Outdated
Comment thread chapters/ai-tools/ai-harness-landscape.qmd
Copilot AI review requested due to automatic review settings September 1, 2026 20:14

Copy link
Copy Markdown
Collaborator Author

Copilot round dispositions (head 7b939f9):

  • Rebutted: the two agent-orchestration.qmd#sec-orch-when links. The fragment is included from chapters/coding-agents.qmd, so links resolve from chapters/, matching the sibling fragments; the link checker passed on this head.
  • Rebutted: the ampcode.com lychee exclusion. The link checker passed with this exclusion and the bare-host link in place.
  • Addressed: the Pi citation's date label now states the post's own date (2025-11-30). Rebutted: the "2nd place" figure comes from issue summarize the current state of AI harnesses #95's summary, which was corrected on the issue after reading the author's post; the chapter's "eighth" is the sourced figure.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are documentation-focused and low-risk, with only minor maintainability nits noted.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

inst/WORDLIST:13

  • auditability and auditable were added in the capitalized proper-noun section, but the WORDLIST is otherwise grouped with capitalized terms first and common lowercase terms later (e.g., agentic, aider, allowlist, ...). Keeping lowercase words in the lowercase section makes the list easier to scan and reduces avoidable merge churn.
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread lychee.toml Outdated
Copilot AI review requested due to automatic review settings September 1, 2026 20:29

Copy link
Copy Markdown
Collaborator Author

Copilot round 2 dispositions (head e2c85cd):

  • Addressed: the Amp lychee exclusion is anchored and escaped (^https://ampcode\.com/?$).
  • Addressed (suppressed nit): auditability and auditable moved out of the capitalized block into the lowercase entries.

Posted by Claude Code (AI agent) --- not written by a human.


Generated by Claude Code

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are documentation-focused and consistent with existing cross-references, with only minor wording/grammar nits noted.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

chapters/ai-tools/ai-harness-landscape.qmd:16

  • In the snapshot callout, this sentence mixes present and past tense ("arrive" vs "changed"/"reshuffled"), which reads like a grammar error. Consider keeping the tense consistent throughout the list.
    chapters/ai-tools/ai-harness-landscape.qmd:180
  • This sentence is ungrammatical: "pain points toward" should be something like "pain points ... point toward" (or rephrase to avoid the double use of "points").
  • Files reviewed: 4/4 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

…loses #95)

Adds a coding-agents section mapping the harness landscape as of August 2026:
the model-plus-harness framing and benchmark saturation, the terminal,
editor, and autonomous tiers, the minimalist counter-trend, the open-source
versus proprietary split, local-model stacks, the general agent-framework
layer, and a job-to-harness selection table. The section is explicitly dated
and attributed to the research summary in issue #95.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE
Restores the source's hedges (repeatedly, some users, Ronacher alone),
separates the two subscription friction points, dates the Gemini CLI
cutoff, the Devin Desktop rebrand, and the Pi benchmark run, links every
table row's project, aligns the Claude Code link with the platforms table,
replaces a forward reference with a table crossref, turns the two product
lists into bullets, and defers to the harness and orchestration sections
instead of restating them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE
…lude ampcode.com from lychee

The link checker reported permanent redirects for Pi, Goose, Kilo Code,
OpenHands, Continue, and Cursor, so the links now name the current
locations, and an HTTP/2 protocol error from ampcode.com, which is the
server's answer to the probe rather than a dead page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE
…te the author's post

The research summary in #95 said second; the author's own December
2025 write-up, which the review checked, says eighth.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE
The link text said "December 2025" while the cited post's slug is
2025-11-30; the post date is the one thing the URL itself verifies, so
the label now states that instead of a month the source does not carry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE
Copilot AI review requested due to automatic review settings September 1, 2026 20:57
@d-morrison
d-morrison force-pushed the docs/ai-harness-landscape branch from e2c85cd to 18b92be Compare September 1, 2026 20:57
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

…words in the lowercase section

The exclusion is a regex, so the bare form matched any URL containing
that string; the anchored, escaped form matches the Amp homepage with or
without its trailing slash and nothing else, like the entry above it.
`auditability` and `auditable` move out of the capitalized block into the
lowercase one where the sibling entries sit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

inst/WORDLIST contains unresolved Git merge-conflict markers that must be fixed before it can be safely merged.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

inst/WORDLIST:136

  • Unresolved Git merge-conflict markers remain in the WORDLIST here as well. Remove the conflict markers and keep both sets of words so spellcheck can run and the list is valid.
<<<<<<< HEAD
Zsh
=======
Zechner
auditability
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread inst/WORDLIST Outdated
Copilot AI review requested due to automatic review settings September 1, 2026 21:05
@d-morrison
d-morrison force-pushed the docs/ai-harness-landscape branch from 18b92be to ad13fac Compare September 1, 2026 21:05
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The documentation addition is internally consistent, integrates cleanly into the chapter structure, and includes the necessary supporting link/spell-check updates.

Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

👀 Claude Review Dispatchedrun reviewing PR #174 at ad13fac.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


Everything checks out cleanly: no conflict markers anywhere, all @sec-* cross-reference targets exist, and no duplicate WORDLIST entries. This confirms the current head commit is byte-identical in content to what was reviewed and marked CLEAN in the last round.

Since the current head commit (ad13fac5878a3a7f92fdcee3943052a9518e8722) is exactly the same commit already reviewed and confirmed CLEAN in the 2026-09-01T21:07:27Z round ("Ready for merge — the blocking finding from the prior round..."), I re-verified the key facts independently rather than just trusting that summary:

  • No merge-conflict markers anywhere in the tree (the previously-blocking issue in inst/WORDLIST stays fixed).
  • All @sec-* cross-references in the new chapter (@sec-ai-harnesses, @sec-ai-coding-agent-platforms, @sec-ai-claude-code-other-models, @sec-ai-offline, @sec-ai-small-local-models, #sec-orch-when) resolve to labels that exist in chapters/coding-agents.qmd and chapters/agent-orchestration.qmd.
  • inst/WORDLIST has no duplicate entries after the additions.
  • lychee.toml's anchored ^https://ampcode\.com/?$ exclusion is unchanged from what was already reviewed and confirmed correct (regex-anchored per the file's own established convention).

No new commits exist on the PR since that round — gh pr view confirms headRefOid still equals ad13fac, the exact commit already reviewed. No new issues found.

Verdict

Ready for merge — head commit ad13fac5878a3a7f92fdcee3943052a9518e8722 is unchanged from the prior CLEAN round (2026-09-01T21:07:27Z), and independent re-verification of conflict markers, cross-reference resolution, and WORDLIST integrity confirms that verdict still stands.

Structured Review Data (JSON)
{
  "schema_version": "1.0",
  "reviewer": "claude",
  "commit_sha": "ad13fac5878a3a7f92fdcee3943052a9518e8722",
  "verdict": "CLEAN",
  "findings": []
}

Reviewed commit: ad13fac

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

💰 Cost: $0.2877 (review) — run

@d-morrison
d-morrison merged commit 46687f3 into main Sep 1, 2026
14 checks passed
@d-morrison
d-morrison deleted the docs/ai-harness-landscape branch September 1, 2026 21:31
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-01 21:31 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

summarize the current state of AI harnesses

3 participants